After-recording apparatus

ABSTRACT

An audio signal is compressively encoded into encoding-resultant audio data. A video signal is compressively encoded into encoding-resultant video data. An audio time stamp for audio-vide synchronous reproduction is added to every unit of the encoding-resultant audio data. A video time stamp for audio-video synchronous reproduction is added to every unit of the encoding-resultant video data. The time-stamp-added audio data and the time-stamp-added video data are multiplexed into main data. To a plurality of first after-recording-purpose data for at least one of (1) the encoding-resultant audio data and (2) the encoding-resultant video data which form the main data, time stamps for reproduction synchronous with a portion of the main data and identification information for identifying the plurality of first after-recording-purpose data are added to convert the first after-recording-purpose data into second after-recording-purpose data. The second after-recording-purpose data are made into bit streams without being multiplexed with the main data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an after-recording apparatus or a dubbing-inapparatus which is designed to post-record, for example, an audio signalin connection with main data including audio data and video data. Inaddition, this invention relates to a computer program, a recordingmedium, a transmission method, and a reproducing apparatus.

2. Description of the Related Art

Japanese patent application publication number 11-144378/1999 disclosesa method of after-recording (post-recording) in which original dataincluding a video bit stream are read out from a digital recordingmedium. The read-out original data are decoded. Audio data are encodedinto a new audio bit stream in synchronism with the decoding of theoriginal data. The new audio bit stream is written into an area of thedigital recording medium which approximately corresponds in timeposition to the original-data recording area.

Japanese patent application publication number 11-259992/1999 disclosesan optical disc on which the post-recording (the after-recording) of newdata can be implemented. The optical disc in Japanese application11-259992 stores a stream of packs including normal packs and reservedpacks. The normal packs are loaded with original moving-picture data.Initially, the reserved packs are unoccupied. New data such as audiodata can be written into the reserved packs. The original moving-picturedata and the new data can be simultaneously reproduced from the opticaldisc.

Japanese patent application publication number P20005-197005A disclosesan information recording medium having an area “A” and an area “B”. Thearea “A” stores a stream of packs including video packs and audio packs.The area “B” stores a table having first, second, and third informationpieces. By referring to the first information piece in the table, adecision is made as to whether or not audio data in the audio packs inthe area “A” correspond to silence. In the case where the audio data inthe audio packs correspond to silence, post-recording (after-recording)can be implemented as follows. By referring to the second and thirdinformation pieces in the table, a new audio signal is encoded into newaudio data, and the new audio data are formatted into new audio packs.The new audio packs are written over the old audio packs in the area“A”.

Recently, multiplexed data containing video data and audio data havesometimes been handled as a form of a transport stream or a programstream. Generally, it is difficult to replace only audio data in suchmultiplexed data with new data.

Prior-art post-recording (prior-art after-recording) writes new audiodata so that an original multiplexed main stream is converted into a newone. Generally, it is difficult to convert the new multiplexed mainstream back into the original one.

In the prior art, it is difficult to post-record a plurality of newaudio signals, and to select one from the new audio signals duringplayback.

SUMMARY OF THE INVENTION

It is a first object of this invention to provide an after-recordingapparatus (a post-recording apparatus or a dubbing-in apparatus) whichenables a post-recording-resultant signal to be converted back into anoriginal signal.

It is a second object of this invention to provide an after-recordingapparatus (a post-recording apparatus or a dubbing-in apparatus) whichcan post-record a plurality of new audio signals, and which enables oneto be selected from the new audio signals during playback.

It is a third object of this invention to provide an improved computerprogram.

It is a fourth object of this invention to provide an improved recordingmedium.

It is a fifth object of this invention to provide an improvedtransmission method.

It is a sixth object of this invention to provide an improvedreproducing apparatus.

A first aspect of this invention provides an after-recording apparatuscomprising first means for compressively encoding an audio signal intoencoding-resultant audio data; second means for compressively encoding avideo signal into encoding-resultant video data; third means for addingan audio time stamp for audio-vide synchronous reproduction to everyunit of the encoding-resultant audio data generated by the first means;fourth means for adding a video time stamp for audio-video synchronousreproduction to every unit of the encoding-resultant video datagenerated by the second means; fifth means for multiplexing thetime-stamp-added audio data generated by the third means and thetime-stamp-added video data generated by the fourth means into maindata; sixth means for, to a plurality of first after-recording-purposeencoding-resultant data for at least one of (1) the encoding-resultantaudio data generated by the first means and (2) the encoding-resultantvideo data generated by the second means which form the main data,adding time stamps for reproduction synchronous with a portion of themain data and identification information for identifying the pluralityof first after-recording-purpose encoding-resultant data to convert thefirst after-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and seventh means formaking the second after-recording-purpose encoding-resultant datagenerated by the sixth means into bit streams without multiplexing thesecond after-recording-purpose encoding-resultant data with the maindata.

A second aspect of this invention provides a computer program for afterrecording. The computer program comprises the steps of compressivelyencoding an audio signal into encoding-resultant audio data;compressively encoding a video signal into encoding-resultant videodata; adding an audio time stamp for audio-vide synchronous reproductionto every unit of the encoding-resultant audio data; adding a video timestamp for audio-video synchronous reproduction to every unit of theencoding-resultant video data; multiplexing the time-stamp-added audiodata and the time-stamp-added video data into main data; to a pluralityof first after-recording-purpose encoding-resultant data for at leastone of (1) the encoding-resultant audio data and (2) theencoding-resultant video data which form the main data, adding timestamps for reproduction synchronous with a portion of the main data andidentification information for identifying the plurality of firstafter-recording-purpose encoding-resultant data to convert the firstafter-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and making the secondafter-recording-purpose encoding-resultant data into bit streams withoutmultiplexing the second after-recording-purpose encoding-resultant datawith the main data.

A third aspect of this invention provides a recording medium on whichafter-recording-purpose data are recorded in a process comprising thesteps of compressively encoding an audio signal into encoding-resultantaudio data; compressively encoding a video signal intoencoding-resultant video data; adding an audio time stamp for audio-videsynchronous reproduction to every unit of the encoding-resultant audiodata; adding a video time stamp for audio-vide synchronous reproductionto every unit of the encoding-resultant video data; multiplexing thetime-stamp-added audio data and the time-stamp-added video data intomain data; to a plurality of first after-recording-purposeencoding-resultant data for at least one of (1) the encoding-resultantaudio data and (2) the encoding-resultant video data which form the maindata, adding time stamps for reproduction synchronous with a portion ofthe main data and identification information for identifying theplurality of first after-recording-purpose encoding-resultant data toconvert the first after-recording-purpose encoding-resultant data intosecond after-recording-purpose encoding-resultant data; making thesecond after-recording-purpose encoding-resultant data into bit streamswithout multiplexing the second after-recording-purposeencoding-resultant data with the main data; and recording the bitstreams on the recording medium.

A fourth aspect of this invention provides a method of transmission. Themethod comprises the steps of compressively encoding an audio signalinto encoding-resultant audio data; compressively encoding a videosignal into encoding-resultant video data; adding an audio time stampfor audio-vide synchronous reproduction to every unit of theencoding-resultant audio data; adding a video time stamp for audio-videosynchronous reproduction to every unit of the encoding-resultant videodata; multiplexing the time-stamp-added audio data and thetime-stamp-added video data into main data; to a plurality of firstafter-recording-purpose encoding-resultant data for at least one of (1)the encoding-resultant audio data and (2) the encoding-resultant videodata which form the main data, adding time stamps for reproductionsynchronous with a portion of the main data and identificationinformation for identifying the plurality of firstafter-recording-purpose encoding-resultant data to convert the firstafter-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; making the secondafter-recording-purpose encoding-resultant data into bit streams withoutmultiplexing the second after-recording-purpose encoding-resultant datawith the main data; and transmitting the bit streams via a transmissionline.

A fifth aspect of this invention provides a reproducing apparatus forreproducing after-recording-purpose data and a portion of main datawhich are generated in a process comprising the steps of compressivelyencoding an audio signal into encoding-resultant audio data;compressively encoding a video signal into encoding-resultant videodata; adding an audio time stamp for audio-vide synchronous reproductionto every unit of the encoding-resultant audio data; adding a video timestamp for audio-video synchronous reproduction to every unit of theencoding-resultant video data; multiplexing the time-stamp-added audiodata and the time-stamp-added video data into main data; recording themain data on a recording medium; to a plurality of firstafter-recording-purpose encoding-resultant data for at least one of (1)the encoding-resultant audio data and (2) the encoding-resultant videodata which form the main data, adding time stamps for reproductionsynchronous with the portion of the main data and identificationinformation for identifying the plurality of firstafter-recording-purpose encoding-resultant data to convert the firstafter-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and recording thesecond after-recording-purpose encoding-resultant data on the recordingmedium without multiplexing the second after-recording-purposeencoding-resultant data with the main data. The reproducing apparatuscomprises first means for selecting one from a plurality of the secondafter-recording-purpose encoding-resultant data in response to user'srequest; second means for reading out the second after-recording-purposeencoding-resultant data selected by the first means from the recordingmedium in response to the identification information correspondingthereto; third means for reading out the main data from the recordingmedium; fourth means for replacing a portion of the main data with thesecond after-recording-purpose encoding-resultant data read out by thesecond means; and fifth means for synchronously reproducing the read-outsecond after-recording-purpose encoding-resultant data and the main dataexcept the portion in response to time stamps therein.

A sixth aspect of this invention provides a computer program forreproducing after-recording-purpose data and a portion of main datawhich are generated in a process comprising the steps of compressivelyencoding an audio signal into encoding-resultant audio data;compressively encoding a video signal into encoding-resultant videodata; adding an audio time stamp for audio-vide synchronous reproductionto every unit of the encoding-resultant audio data; adding a video timestamp for audio-video synchronous reproduction to every unit of theencoding-resultant video data; multiplexing the time-stamp-added audiodata and the time-stamp-added video data into main data; recording themain data on a recording medium; to a plurality of firstafter-recording-purpose encoding-resultant data for at least one of (1)the encoding-resultant audio data and (2) the encoding-resultant videodata which form the main data, adding time stamps for reproductionsynchronous with the portion of the main data and identificationinformation for identifying the plurality of firstafter-recording-purpose encoding-resultant data to convert the firstafter-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and recording thesecond after-recording-purpose encoding-resultant data on the recordingmedium without multiplexing the second after-recording-purposeencoding-resultant data with the main data. The computer programcomprises the steps of selecting one from a plurality of the secondafter-recording-purpose encoding-resultant data in response to user'srequest; reading out the selected second after-recording-purposeencoding-resultant data from the recording medium in response to theidentification information corresponding thereto; reading out the maindata from the recording medium; replacing a portion of the main datawith the read-out second after-recording-purpose encoding-resultantdata; and synchronously reproducing the readout secondafter-recording-purpose encoding-resultant data and the main data exceptthe portion in response to time stamps therein.

A seventh aspect of this invention provides an after-recording apparatuscomprising first means for encoding a first audio signal into firstencoding-resultant audio data; second means for encoding a video signalinto encoding-resultant video data; third means for adding a first audiotime stamp for audio-vide synchronous playback to the firstencoding-resultant audio data generated by the first means; fourth meansfor adding a video time stamp for audio-video synchronous playback tothe encoding-resultant video data generated by the second means; fifthmeans for multiplexing the time-stamp-added audio data generated by thethird means and the time-stamp-added video data generated by the fourthmeans into main data; sixth means for recording the main data generatedby the fifth means on a recording medium; seventh means for encoding asecond audio signal into second encoding-resultant audio data; eighthmeans for encoding a third audio signal into third encoding-resultantaudio data; ninth means for adding a second audio time stamp foraudio-vide synchronous playback and first identification information tothe second encoding-resultant audio data generated by the seventh means,the second audio time stamp being equivalent to the first audio timestamp regarding audio-vide synchronous playback; tenth means for addinga third audio time stamp for audio-vide synchronous reproduction andsecond identification information to the third encoding-resultant audiodata generated by the eighth means, the third audio time stamp beingequivalent to the first audio time stamp regarding audio-videsynchronous playback, the second identification information beingdifferent from the first identification information; and eleventh meansfor recording the time-stamp-added and identification-information-addedaudio data generated by the ninth means and the time-stamp-added andidentification-information-added audio data generated by the tenth meanson the recording medium.

An eighth aspect of this invention provides a reproducing apparatus foruse with a recording medium storing main data and a plurality ofafter-recording-purpose audio data, the main data including main videodata and main audio data, the main video data containing a video timestamp, the main audio data containing a first audio time stamp, thevideo time stamp and the first audio time stamp being for synchronousplayback of video and audio, the plurality of after-recording-purposeaudio data having different identification information piecesrespectively, the plurality of after-recording-purpose audio datacontaining second audio time stamps respectively, the second audio timestamps being equivalent to the first audio time stamp regardingsynchronous playback of video and audio. The reproducing apparatuscomprises first means for designating desired one among the plurality ofafter-recording-purpose audio data in response to user's request; secondmeans for deciding an identification information piece corresponding tothe desired after-recording-purpose audio data designated by the firstmeans; third means for reading out the desired after-recording-purposeaudio data from the recording medium in response to the identificationinformation piece decided by the second means; fourth means for readingout the main data from the recording medium; fifth means for separatingthe main data read out by the fourth means into the main video data andthe main audio data; sixth means for detecting the video time stamp inthe main video data generated by the fifth means; seventh means fordetecting a second audio time stamp in the desiredafter-recording-purpose audio data read out by the third means; andeighth means for synchronously playing back the desiredafter-recording-purpose audio data read out by the third means and themain video data generated by the fifth means in response to the videotime stamp detected by the sixth means and the second audio time stampdetected by the seventh means.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an MPEG encoder.

FIG. 2 is a block diagram of an MPEG decoder.

FIG. 3 is a block diagram of an MPEG-system decoding apparatus.

FIG. 4 is a diagram of the relation among an MPEG transport stream (TS),an MPEG program stream (PS), and packetized elementary streams (PESs).

FIG. 5 is a diagram of the format of a TS packet.

FIG. 6 is a flow diagram of the reception of TS packets in response topacket IDs (PIDs).

FIG. 7 is a block diagram of an after-recording apparatus in a firstembodiment of this invention.

FIG. 8 is a diagram of the file structure of library informationrecorded on a recording medium.

FIG. 9 is a diagram of the structure of a file “SIDE.ifo” in FIG. 8.

FIG. 10 is a diagram of the syntax structure of a second-level segment“GENERAL_IFO” in FIG. 9.

FIG. 11 is a diagram of the structure of a third-level segment“PR_IFO_(—)0” in FIG. 9.

FIG. 12 is a diagram of the syntax structure of a fourth-level segment“PROG_IFO” in FIG. 11.

FIG. 13 is a diagram of the syntax structure of play list information“PLAYL_IFO” in FIG. 9.

FIG. 14 is a diagram of the syntax structure of index information“INDEX_IFO” in FIG. 11.

FIG. 15 is a diagram of the format of a transport stream (TS) recordedon a hard disk.

FIG. 16 is a block diagram of a reproducing apparatus in the firstembodiment of this invention.

FIG. 17 is a block diagram of a transmission apparatus in the firstembodiment of this invention.

FIG. 18 is a flowchart of a segment of a control program for a CPU inFIG. 7.

FIG. 19 is a flowchart of a segment of a control program for a CPU inFIG. 16.

DETAILED DESCRIPTION OF THE INVENTION First Embodiment

FIG. 1 shows an MPEG (Moving Picture Experts Group) encoder whichincludes an adder 1 operated as a subtracter. The subtracter 1calculates an inter-frame prediction error, that is, a residual (anerror) between a picture represented by an input video signal Vin and afinal reference picture (a final predicted picture) represented byoutput data from a motion-compensated predictor 11. The subtracter 1outputs data indicating the calculated residual (the calculatedinter-frame prediction error).

Prediction implemented by the motion-compensated predictor 11 isdesigned as follows. Prediction directions are of three modes, that is,a direction from the past (“forward”), a direction from the future(“backward”), and a direction from both the past and future(“interpolative”). Accordingly, there are prediction from the past(“forward”), prediction from the future (“backward”), and predictionfrom both the past and future (“interpolative”). An actually-usedprediction direction can be changed MB by MB, where MB denotes amacro-block having 16 by 16 pixels. Every picture is divided into aprescribed number of MBs. The actually-used prediction direction isbasically decided by the type of every picture represented by the inputvideo signal Vin. Pictures are classified into P pictures (predictivecoded pictures), B pictures (bidirectionally predictive coded pictures),and I pictures (intra-coded pictures). MBs representative of a P pictureare of first and second modes. P-picture MBs of the first mode areencoded according to prediction from the past. P-picture MBs of thesecond mode are independently encoded without prediction. MBsrepresentative of a B picture are of first, second, third, and fourthmodes. B-picture MBs of the first mode are encoded according toprediction from the future. B-picture MBs of the second mode are encodedaccording to prediction from the past. B-picture MBs of the third modeare encoded according to prediction from both the past and future.B-picture MBs of the fourth mode are independently encoded withoutprediction. MBs representative of an I picture are independently encodedwithout prediction.

Motion compensation implemented by the motion-compensated predictor 11is designed as follows. According to motion compensation, patternmatching between two pictures is performed for each MB to detect amotion vector or motion vectors at an accuracy corresponding to a halfpel (a half pixel). One of the two pictures is given by the input videosignal Vin, and the other is referred to as a basic reference picture ora source picture for motion-compensated prediction. The basic referencepicture is shifted in accordance with detected motion vectors. A finalreference picture (a final predicted picture) is generated on the basisof the shift of the basic reference picture. In the absence of motioncompensation, the basic reference picture is directly used as a finalreference picture. Generally, a motion vector has a horizontal-directioncomponent and a vertical-direction component. Information representing adetected motion vector or detected motion vectors, and MC-mode(motion-compensation mode) information representing a predictiondirection or indicating a source picture from which prediction isimplemented are transmitted as added information relating to each MB. AGOP (group of pictures) is defined as a sequence of pictures startingfrom an I picture and ending at a picture immediately preceding a next Ipicture. Generally, one GOP has about 15 successive pictures.

The residual-indicating data outputted from the subtracter 1 aresubjected to discrete cosine transform (DCT) by a DCT device 2.Specifically, the DCT device 2 divides every MB represented by theresidual-indicating data into 4 DCT blocks each having 8 by 8 pixels.

The DCT device 2 subjects each DCT block to two-dimensional DCT togenerate data representing DCT coefficients. The DCT device 2 outputsthe generated DCT-coefficient data. In general, since a video signal isrich in low-frequency components, there occur a lot of effective DCTcoefficients corresponding to low frequencies.

DCT coefficients represented by the output data from the DCT device 2are quantized by a quantizer 3 in response to a quantization value. Thequantization value is equal to a quantization matrix multiplied by ascalar quantization scale. The quantization matrix has 8 by 8 elementsresulting from a process of weighting two-dimensional frequencycomponents according to visual sensation. The quantization includes astep of dividing DCT coefficients by the quantization value.

Quantization-resultant data outputted from the quantizer 3 are encodedby a VLC device 4 into data of a variable length code (VLC).Specifically, a direct-current (DC) component of thequantization-resultant data is encoded by DPCM (differential pulse codemodulation). Alternating-current (AC) components of thequantization-resultant data are scanned in zigzag along a direction froma high frequency toward a low frequency, and are subjected to Huffmancoding in which data pieces having higher occurrence possibilities areassigned to shorter code words. The VLC device 4 receives themotion-vector information and the MC-mode information from themotion-compensated predictor 11. The VLC device 4 adds the motion-vectorinformation and the MC-mode information to the VLC data. The resultantVLC data are temporarily stored in a buffer 5. The VLC data areoutputted from the buffer 5 at a prescribed transfer rate asencoding-resultant MPEG data in the form of a bit stream.

The buffer 5 informs a code amount controller 6 of the actual amount ofencoding-resultant data for every MB, that is, the total number of bitscomposing encoding-resultant data for every MB. The code amountcontroller 6 calculates an error between the actual amount ofencoding-resultant data and a target amount thereof. The code amountcontroller 6 adjusts the quantization scale used by the quantizer 3 inresponse to the calculated data amount error, and thereby controls theactual amount of encoding-resultant data.

The quantization-resultant data outputted from the quantizer 3 aresubjected to inverse quantization by an inverse quantizer 7, beingconverted back to DCT-coefficient data. The inverse quantizer 7 outputsthe DCT-coefficient data. The DCT-coefficient data outputted from theinverse quantizer 7 are subjected to inverse DCT by an inverse DCTdevice 8, being converted back into residual-indicating data. Theinverse DCT device 8 outputs the residual-indicating data. A residualpicture represented by the output data from the inverse DCT device 8 anda final reference picture (a final predicted picture) represented by theoutput data from the motion-compensated predictor 11 are added by anadder 9. The adder 9 outputs addition-resultant video data. Theaddition-resultant video data are temporarily stored in a memory 10.Video data are fed from the memory 10 to the motion-compensatedpredictor 11. The output video data from the memory 10 are used by themotion-compensated predictor 11 as an indication of a basic referencepicture or a source picture for motion-compensated prediction.

FIG. 2 shows an MPEG decoder which includes a buffer 12.Encoding-resultant MPEG data in the form of a bit stream are temporarilystored in the buffer 12 before being outputted to a VLD (variable lengthdecoding) device 13 therefrom. The VLD device 13 subjects the outputdata from the buffer 12 to VLD inverse with respect to VLC, therebyrecovering DC-component data and AC-component data. The AC-componentdata are scanned in zigzag along a direction from a high frequencytoward a low frequency. The DC-component data and the resultantAC-component data are arranged in a matrix having 8 by 8 elements. TheVLD device 13 outputs the matrix-arranged data to an inverse quantizer14. The VLD device 13 extracts motion-vector information and MC-modeinformation from the data outputted by the buffer 12. The VLD device 13outputs the motion-vector information and the MC-mode information to amotion-compensated predictor 18.

The inverse quantizer 14 subjects the output data from the VLD device 13to inverse quantization responsive to the quantization value, therebyrecovering DCT-coefficient data. The inverse quantization includes astep of multiplying processed data values by the quantization value. Itshould be noted that the quantization value is equal to the quantizationmatrix multiplied by the scalar quantization scale. The DCT-coefficientdata are outputted from the inverse quantizer 14 to an inverse DCTdevice 15, being subjected to inverse DCT and hence being converted backinto residual-indicating data. The inverse DCT device 15 outputs theresidual-indicating data. A residual picture represented by the outputdata from the inverse DCT device 15 and a final reference picture (afinal predicted picture) represented by output data from themotion-compensated predictor 18 are added by an adder 16. The adder 16outputs addition-resultant video data as decoding-resultant video data(original video data). The decoding-resultant video data are transmittedtoward an external. The addition-resultant video data outputted from theadder 16 are temporarily stored in a memory 17. Video data are fed fromthe memory 17 to the motion-compensated predictor 18. The output videodata from the memory 17 are used by the motion-compensated predictor 18as an indication of a basic reference picture or a source picture formotion-compensated prediction. The motion-compensated predictor 18generates a final reference picture (a final predicted picture) inresponse to the basic reference picture, the motion-vector information,and the MC-mode information. The motion-compensated predictor 18 outputsdata representing the final reference picture.

An MPEG system will be explained below. The MPEG system prescribes thefollowing five items.

-   1) Synchronous reproduction of a plurality of encoding-resultant bit    streams;-   2) Multiplexing a plurality of encoding-resultant bit streams into a    single bit stream;-   3) Initialization of a buffer at start of reproduction;-   4) Continuous management of a buffer; and-   5) Identification of time of decoding or reproduction.

The multiplexing of information in the MPEG system includes a step ofpacketing the information. For example, in the case where videoinformation and audio information are required to be multiplexed, eachof the video information and the audio information is divided intopackets having a suitable length. Additional information such as headerinformation is added to each packet. Video-information packets andaudio-information packets are mixed into a packet sequence before thepacket sequence is transmitted. Therefore, the video-information packetsand the audio-information packets are transmitted on a time sharingbasis. The header information contains information for identifyingwhether a related packet is of video or audio, and timing informationfor synchronization. In general, the packet length depends on atransmission medium and an application. For example, the packet lengthis equal to 53 bytes in the case of ATM (asynchronous transfer mode).The packet length is equal to 4 kilobytes in the case of an opticaldisc. According to the MPEG system, the packet length is variable, andcan be set to an arbitrary value.

Data to be transmitted are divided into packs, and are packeted. Onepack is composed of several packets. A header portion of each pack isloaded with a pack-start-code and an SCR (system clock reference). Aheader of each packet is loaded with a stream ID and a time stamp. Thetime stamp contains timing information for synchronization between audioand video. The time stamp is of two types referred to as a DTS (decodingtime stamp) and a PTS (presentation time stamp) respectively. Thetransmitted data contain a periodically-occurring PCR (program clockreference) indicating a frequency of 27 MHz. A reference clock common todecoders can be locked to the frequency indicated by the PCR. The DTSdenotes the desired decoding start time for a first access unit in arelated packet, while the PTS denotes the desired indication start time(the desired playback start time) therefor. One access unit correspondsto one picture in the case of video. One access unit corresponds to 1152samples in the case of audio. The DTS and the PTS are given at a timeaccuracy based on the frequency indicated by the PCR.

FIG. 3 shows an MPEG-system decoding apparatus which includes a systemdecoder 20 receiving multiplexed data (a system bit stream) containingvideo information and audio information. The system decoder 20demultiplexes the received data into video data and audio data. Thesystem decoder 20 outputs the video data to a video decoder (an MPEGvideo decoder) 21. The system decoder 20 outputs the audio data to anaudio decoder (an MPEG audio decoder) 22. In addition, the systemdecoder 20 extracts every PCR, every video DTS, every audio DTS, everyvideo PTS, and every audio PTS from the received data. The systemdecoder 20 may extract every SCR from the received data. The systemdecoder 20 outputs the PCR (or the SCR) to a clock control circuit 23.The system decoder 20 outputs the video DTS, the audio DTS, the videoPTS, and the audio PTS to a comparison circuit 24.

The clock control circuit 23 includes an oscillator for generating areference clock signal denoting reference clock time. The clock controlcircuit 23 locks the frequency of oscillation of the oscillator to afrequency indicated by the PCR. Therefore, the reference clock time isdecided on the basis of the PCR. The clock control circuit 23 informsthe comparison circuit 24 of reference clock time, that is, time denotedby the reference clock signal.

The comparison circuit 24 compares time denoted by the video DTS withthe reference clock time to decide whether or not the video DTS timeagrees with the reference clock time. When the video DTS time agreeswith the reference clock time, the comparison circuit 24 enables thevideo decoder 21 to start the decoding of the video data. The videodecoder 21 stores the decoding-resultant video data into a memory 25.The comparison circuit 24 compares time denoted by the audio DTS withthe reference clock time to decide whether or not the audio DTS timeagrees with the reference clock time. When the audio DTS time agreeswith the reference clock time, the comparison circuit 24 enables theaudio decoder 22 to start the decoding of the audio data. The audiodecoder 21 stores the decoding-resultant audio data into a memory 26.

The comparison circuit 24 compares time denoted by the video PTS withthe reference clock time to decide whether or not the video PTS timeagrees with the reference clock time. When the video PTS time agreeswith the reference clock time, the comparison circuit 24 enables thememory 25 to output the decoding-resultant video data for an indicationpurpose. The comparison circuit 24 compares time denoted by the audioPTS with the reference clock time to decide whether or not the audio PTStime agrees with the reference clock time. When the audio PTS timeagrees with the reference clock time, the comparison circuit 24 enablesthe memory 26 to output the decoding-resultant audio data for a playbackpurpose. The video PTS and the audio PTS are preset so that thedecoding-resultant video data and the decoding-resultant audio data canbe synchronously outputted from the memories 25 and 26. Therefore, videoinformation and audio information can be synchronously played back.

It is considered that a portion of the MPEG-system decoding apparatus ofFIG. 3 corresponds to a virtual decoder for temporarily storingdecoding-resultant data of plural types, and implementing synchronousreproduction of the plural-type data. The virtual decoder is referred toas an STD (system target decoder). A multiplexing-resultant bit streamis designed so that memories in the STD will neither overflow norunderflow.

According to the MPEG system, there are a transport stream (TS) and aprogram stream (PS). The TS or the PS is formed by packetized elementarystreams (PESs) and packets loaded with other information. The PESs aredefined as intermediate streams for conversion or transformation betweena TS and a PS. The PESs are generated by packeting, for example,encoding-resultant MPEG video data, encoding-resultant MPEG audio data,or a private stream.

Video and audio data of content programs having common reference timecan be multiplexed into a PS. The PS includes a sequence of packets. Apacket layer of the PS is called “PES”. With reference to FIG. 4, thepacket layer of the PS is common to that of a TS so that compatibilitybetween the PS and the TS is provided. According to an STD model for aPS, the decoding mode for a bit stream to be decoded is switched to a PSside by a stream ID in a PES packet.

Video and audio data of content programs having common reference timecan also be multiplexed into a TS. Furthermore, video and audio data ofcontent programs different in reference time can be multiplexed into aTS. The TS is formed by a sequence of fixed-length packets, that is,188-byte packets (TS packets). The TS is designed to be usable in asystem where a data error or data errors are caused by a transmissionline. As understood from the above explanation, the TS is a streamrepresenting multiple content-programs. Although a TS packet rankshigher than a PES packet, the TS packet is normally shorter than the PESpacket. Generally, one PES packet is divided into segments, and the PESpacket segments are placed in plural TS packets respectively. Accordingto an STD model for a TS, the decoding mode for a bit stream to bedecoded is switched to a PS side by a packet ID (PID) in a TS packet.

With reference to FIG. 5, a TS packet has a header, an adaptation field(AF), and a payload which are sequentially arranged in that order. Theheader includes a sync byte, an error indicator, a unit start indicator,and a transport packet priority flag which are sequentially arranged inthat order. In the header, the transport packet priority flag isfollowed by a PID which indicates the type of the payload in the relatedpacket (the type of data in the payload in the related packet). In theheader, the PID is successively followed by a scramble controlinformation piece, an AF control information piece, and a cycliccounter. The tail of the header is occupied by the cyclic counter. TheAF control information piece represents whether the adaptation filed(AF) is present in or absent from the related packet. When the AFcontrol information piece represents the absence of the adaptationfield, the header is immediately followed by the payload. On the otherhand, when the AF control information piece represents the presence ofthe adaptation field, the header is successively followed by theadaptation field (AF) and the payload. The cyclic counter indicates thecontinuity about the related packet. In general, adaptation informationis placed in the adaptation field (AF) while content data such as videodata or audio data are placed in the payload. Ineffective data (dummydata) can be placed in the payload.

As shown in FIG. 5, the adaptation field includes an optional field. A48-bit front end of the optional field is loaded with a PCR.

The MPEG system handling a TS is designed to provide a mechanism fordetecting the PIDs in TS packets and classifying the TS packetsaccording to the detected PIDs during reception and decoding procedures.Specifically, with reference to FIG. 6, TS packets are searched for onehaving a PID=0 at a stage S1. In other words, a TS packet having a PID=0is received. The TS packet having a PID=0 is referred to as a PAT(program association table) packet. The PAT packet is loaded with PATinformation representing the relation between program numbers(content-program ID numbers) PR and special PIDs on a link basis, thatis, the relation between content programs and the special PIDs. At astage S2 following the stage S1, the PAT information in the PAT packetis detected. Each of the special PIDs corresponds to a PMT (program maptable) packet. The PMT packet is loaded with PMT informationrepresenting the relation among the related content program, PIDs invideo packets representing the related content program, and PIDs inaudio packets representing the related content program. One is selectedin advance from content programs as a desired content program. At thestage S2, one of the special PIDs which corresponds to the desiredcontent program is detected by referring to the PAT information in thePAT packet. At a stage S3 following the stage S2, a PMT packet having aPID identical with the detected special PID is received or searched for.At a stage S4 subsequent to the stage S3, the received PMT packet isaccessed to get PMT information. According to the PMT information in theaccessed PMT packet, PIDs in video packets and audio packetsrepresenting the desired content program are detected. In addition, PIDsin sync clock packets relating to the desired content program aredetected. At a stage S5 a following the stage S4, TS packetscorresponding to the detected video packet PIDs are received. Videoinformation is extracted from the received TS packets. At a stage S5 bfollowing the stage S4, TS packets corresponding to the detected audiopacket PIDs are received. Audio information is extracted from thereceived TS packets. At a stage S5 c following the stage S4, TS packetscorresponding to the detected sync clock packet PIDs are received. Syncclock information is extracted from the received TS packets. In thisway, the video packets and the audio packets representing the desiredcontent program are accessed in response to the detected PIDs. Entryinto the desired content program is implemented by referring to the PATand the PMT. The PAT and the PMT are called PSI (program specificinformation).

FIG. 7 shows an after-recording apparatus (a post-recording apparatus)using the MPEG encoder of FIG. 1. As shown in FIG. 7, theafter-recording apparatus includes a user interface 109 connected with aCPU 110. The user interface 109 can be handled by a user. The CPU 110has a combination of an input/output port, a processing section, a ROM,and a RAM. The CPU 110 operates in accordance with a control programstored in the ROM or the RAM. The control program is designed to enablethe CPU 110 to implement operation steps mentioned later. By handlingthe user interface 109, operation of the after-recording apparatus canbe changed among different modes including a first mode and a secondmode. The first operation mode is designed to record main data. Thesecond operation mode is designed to record after-recording-purposeaudio data. When the user interface 109 is handled, an operation-modedesignation signal is inputted therefrom into the CPU 110. Theoperation-mode designation signal indicates which of the first operationmode and the second operation mode is desired. The CPU 110 transfers theoperation-mode designation signal to a signal selector 105.

When the operation-mode designation signal indicates that the firstoperation mode is desired, that is, when the recording of main data isdesired, the after-recording apparatus operates as follows. Input videodata are fed to a video encoder 101 a, and input audio data are fed toan audio encoder 101 b. In general, the input video data and the inputaudio data represent a common content program (a common audio-visualprogram), and are synchronous with each other. The video encoder 101 aimplements the MPEG encoding of the input video data to generateencoding-resultant video data. The encoding-resultant video data aresent from the video encoder 101 a to a PES packeting device 102 a. Theaudio encoder 101 b implements the MPEG encoding of the input audio datato generate encoding-resultant audio data. The encoding-resultant audiodata are sent from the audio encoder 101 b to a PES packeting device 102b. The PES packeting device 102 a converts the encoding-resultant videodata into a sequence of PES packets. The PES packet sequence is sentfrom the PES packeting device 102 a to a time stamp recorder 103 a. ThePES packeting device 102 b converts the encoding-resultant audio datainto a sequence of PES packets. The PES packet sequence is sent from thePES packeting device 102 b to a time stamp recorder 103 b.

A signal generator 103 c outputs a 27-MHz clock signal to the time stamprecorders 103 a and 103 b. The time stamp recorder 103 a generatestiming information pieces, that is, a PCR and periodically-updated timestamps (a video PTS and a video DTS), in response to the 27-MHz clocksignal. The time stamp recorder 103 a records the PCR, the PTS, and theDTS in each PES packet. Timing-information-added PES packets aresequentially sent from the time stamp recorder 103 a to a multiplexer104. The time stamp recorder 103 b generates a PCR andperiodically-updated time stamps (an audio PTS and an audio DTS) inresponse to the 27 MHz clock signal. The time stamp recorder 103 brecords the PCR, the PTS, and the DTS in each PES packet.Timing-information-added PES packets are sequentially sent from the timestamp recorder 103 b to the multiplexer 104. The multiplexer 104multiplexes the PES packets from the time stamp recorder 103 a and thePES packets from the time stamp recorder 103 b to generatemultiplexing-resultant data (main data) of a PS form or a TS form. Themultiplexing-resultant data are sent from the multiplexer 104 to thesignal selector 105.

The signal selector 105 selects the multiplexing-resultant data (themain data) in response to the operation-mode designation signal, andpasses the multiplexing-resultant data to a buffer 106 a. Themultiplexing-resultant data are stored in the buffer 106 a before beingoutputted therefrom to a recording controller 107. The recordingcontroller 107 records the main data (the multiplexing-resultant data)on a recording medium 108 as a file having a name “PR . . . .dat” (seeFIG. 8).

The video PTS and the video DTS recorded by the time stamp recorder 103a, and the audio PTS and the audio DTS recorded by the time stamprecorder 103 b are in a relation such that the video information and theaudio information can be synchronously reproduced from the recorded maindata.

When the operation-mode designation signal indicates that the secondoperation mode is desired, that is, when the recording ofafter-recording-purpose audio data is desired, the after-recordingapparatus operates as follows. The CPU 110 sends identificationinformation (ID information) of after-recording-purpose audio data tothe PES packeting device 102 b. After-recording-purpose audio data arefed to the audio encoder 101 b. The after-recording purpose audio datahave a synchronized relation with the video information in the maindata. The audio encoder 101 b implements the MPEG encoding of theafter-recording-purpose audio data to generate encoding-resultant audiodata. The encoding-resultant audio data are sent from the audio encoder101 b to the PES packeting device 102 b.

The PES packeting device 102 b adds the after-recording ID informationto the encoding-resultant audio data, and converts the ID-addedencoding-resultant video data into a sequence of PES packets. The PESpacket sequence is sent from the PES packeting device 102 b to the timestamp recorder 103 b. The signal generator 103 c outputs the 27-MHzclock signal to the time stamp recorder 103 b. The time stamp recorder103 b generates timing-information pieces, that is, a PCR andperiodically-updated time stamps (an audio PTS and an audio DTS), inresponse to the 27-MHz clock signal. The time stamp recorder 103 brecords the PCR, the PTS, and the DTS in each PES packet. Preferably,the PCR, the PTS, and the DTS are set equal to those which have beenadded to each audio PES packet during the recording of the main data. Inthis case, the timing relation of the after-recorded audio data with thevideo information in the recorded main data will be the same as that ofthe audio information in the recorded main data with the videoinformation therein. Timing-information-added PES packets aresequentially sent from the time stamp recorder 103 b to the signalselector 105 without being propagated through the multiplexer 104. Thesignal selector 105 selects the data (the PES packet sequence) outputtedby the time stamp recorder 103 b in response to the operation-modedesignation signal, and passes the selected data to a buffer 106 b. Theselected data are stored in the buffer 106 b before being outputted fromthe buffer 106 b to the recording controller 107. The recordingcontroller 107 records the output data from the buffer 106 b on arecording medium 108 as after-recorded audio data (post-recorded audiodata). The after-recorded audio data on the recording medium 108 are ina file having a name “AF- . . . .dat” (see FIG. 8).

As previously mentioned, the CPU 110 is connected with the userinterface 109. The CPU 110 can receive an operation-mode designationsignal from the user interface 109. The CPU 110 responds to theoperation-mode designation signal. The CPU 110 is connected with thedevices 101 a, 101 b, 102 a, 102 b, 103 a, 103 b, 104, 105, 106 a, 106b, and 107. The CPU 110 can control the devices 101 a, 101 b, 102 a, 102b, 103 a, 103 b, 104, 105, 106 a, 106 b, and 107. As previouslymentioned, the CPU 110 has a combination of an input/output port, aprocessing section, a ROM, and a RAM. The CPU 110 operates in accordancewith a control program stored in the ROM or the RAM. Preferably, thecontrol program is transmitted into the CPU 110 from a recording medium.The control program may be downloaded into the CPU 110 via acommunication network.

FIG. 18 is a flowchart of a segment of the control program for the CPU110. As shown in FIG. 18, a first step S10 of the program segmentimplements the reception of input data (input video data and input audiodata) to be recorded. The step S10 feeds the input video data to thevideo encoder 101 a. The step S10 feeds the input audio data to theaudio encoder 101 b.

A step S11 following the step S10 decides whether or not theafter-recording operation mode (the second operation mode) is desired byreferring to an operation-mode designation signal outputted from theuser interface 109. When the after-recording operation mode is desired,the program advances from the step S11 to a step S12. Otherwise, theprogram advances from the step S11 to a step S13.

The step S13 controls the video encoder 101 a to implement the MPEGencoding of the input video data to generate encoding-resultant videodata. The encoding-resultant video data are sent from the video encoder101 a to the PES packeting device 102 a. The step S13 controls the audioencoder 101 b to implement the MPEG encoding of the input audio data togenerate encoding-resultant audio data. The encoding-resultant audiodata are sent from the audio encoder 101 b to the PES packeting device102 b.

A step S14 subsequent to the step S13 controls the PES packeting device102 a to convert the encoding-resultant video data into a sequence ofPES packets. The PES packet sequence is sent from the PES packetingdevice 102 a to the time stamp recorder 103 a. The step S14 controls thePES packeting device 102 b to convert the encoding-resultant audio datainto a sequence of PES packets. The PES packet sequence is sent from thePES packeting device 102 b to the time stamp recorder 103 b.

A step S15 following the step S14 controls the time stamp recorder 103 ato generate timing information pieces, that is, a PCR andperiodically-updated time stamps (a video PTS and a video DTS), inresponse to the 27-MHz clock signal. The step S15 controls the timestamp recorder 103 a to record the PCR, the PTS, and the DTS in each PESpacket. Timing-information-added PES packets are sequentially sent fromthe time stamp recorder 103 a to the multiplexer 104. The step S15controls the time stamp recorder 103 b to generate a PCR andperiodically-updated time stamps (an audio PTS and an audio DTS) inresponse to the 27-MHz clock signal. The step S15 controls the timestamp recorder 103 b to record the PCR, the PTS, and the DTS in each PESpacket. Timing-information-added PES packets are sequentially sent fromthe time stamp recorder 103 b to the multiplexer 104.

A step S16 subsequent to the step S15 controls the multiplexer 104 tomultiplex the PES packets from the time stamp recorder 103 a and the PESpackets from the time stamp recorder 103 b, and thereby to generatemultiplexing-resultant data (main data) of a PS form or a TS form. Themultiplexing-resultant data are sent from the multiplexer 104 to thesignal selector 105.

A step S17 following the step S16 controls the signal selector 105 inresponse to the operation-mode designation signal so that the signalselector 105 selects the multiplexing-resultant data (the main data)from the multiplexer 104 and passes the multiplexing-resultant data tothe buffer 106 a. The step S17 controls the buffer 106 a to store themultiplexing-resultant data.

A step S18 subsequent to the step S17 monitors the amount (the number ofbits) of the multiplexing-resultant data in the buffer 106 a. The stepS18 decides whether or not the amount of the multiplexing-resultant datain the buffer 106 a exceeds a prescribed amount. When the amount of themultiplexing-resultant data in the buffer 106 a exceeds the prescribedamount, the program advances from the step S18 to a step S19. Otherwise,the program returns from the step S18 to the step S17.

The step S19 controls the buffer 106 a to output themultiplexing-resultant data to the recording controller 107. The stepS19 controls the recording controller 107 to record the main data, thatis, the multiplexing-resultant data, on a recording medium 108 as a filehaving a name “PR . . . .dat” (see FIG. 8).

A step S20 following the step S19 decides whether or not remaining inputdata exist. When remaining input data exist, the program returns fromthe step S20 to the step S10. Otherwise, the program advances from thestep S20, and then the current execution cycle of the program segmentends.

The step S12 controls the audio encoder 101 b to implement the MPEGencoding of the input audio data (the after-recording-purpose audiodata) to generate encoding-resultant audio data. The encoding-resultantaudio data are sent from the audio encoder 101 b to the PES packetingdevice 102 b.

A step S21 subsequent to the step S20 sends identification information(ID information) of the after-recording-purpose audio data to the PESpacketing device 102 b. The step S21 controls the PES packeting device102 b to add the after-recording ID information to theencoding-resultant audio data, and to convert the ID-addedencoding-resultant video data into a sequence of PES packets. The PESpacket sequence is sent from the PES packeting device 102 b to the timestamp recorder 103 b.

A step S22 following the step S21 controls the time stamp recorder 103 bto generate timing-information pieces, that is, a PCR andperiodically-updated time stamps (an audio PTS and an audio DTS), inresponse to the 27-MHz clock signal. The step S22 controls the timestamp recorder 103 b to record the PCR, the PTS, and the DTS in each PESpacket. Preferably, the PCR, the PTS, and the DTS are set equal to thosewhich have been added to each audio PES packet during the recording ofthe main data. In this case, the timing relation of the after-recordedaudio data with the video information in the recorded main data will bethe same as that of the audio information in the recorded main data withthe video information therein. Timing-information-added PES packets aresequentially sent from the time stamp recorder 103 b to the signalselector 105 without being propagated through the multiplexer 104.

The step S17 subsequent to the step S16 controls the signal selector 105in response to the operation-mode designation signal so that the signalselector 105 selects the audio data (the PES packet sequence) outputtedby the time stamp recorder 103 b and passes the selected audio data tothe buffer 106 b. The step S17 controls the buffer 106 b to store theselected audio data.

The step S18 following the step S17 monitors the amount (the number ofbits) of the audio data in the buffer 106 b. The step S18 decideswhether or not the amount of the audio data in the buffer 106 b exceedsa prescribed amount. When the amount of the audio data in the buffer 106b exceeds the prescribed amount, the program advances from the step S18to the step S19. Otherwise, the program returns from the step S18 to thestep S17.

The step S19 controls the buffer 106 b to output the audio data to therecording controller 107. The step S19 controls the recording controller107 to record the audio data from the buffer 106 b on a recording medium108 as after-recorded audio data (post-recorded audio data). Theafter-recorded audio data on the recording medium 108 are in a filehaving a name “AF- . . . .dat” (see FIG. 8). The step S19 is followed bythe step S20.

According to an information format mentioned later,after-recording-purpose audio data are recorded on a recording medium asplay list information. Specifically, first after-recording-purpose audiodata are recorded as a file having a name “AF-1.dat” in a PLO folder(see FIG. 8). Also, second and later after-recording-purpose audio dataare recorded. Furthermore, m-th after-recording-purpose audio data arerecorded as a file having a name “AF-m.dat” in the PLO folder (see FIG.8). Thus, “m” different types of after-recording-purpose audio data arerecorded. An information piece “AF_number” (see FIG. 13) identifiesafter-recording-purpose audio data. Since the information piece“AF_number” has 8 bits (“0” is unused), up to 254 different types ofafter-recording-purpose audio data can be recorded.

The format of information recorded on a recording medium 108 will beexplained below. As shown in FIG. 8, the recording medium 108 stores aROOT directory under which a folder named “LIB (library)” is placed.Under the folder “LIB”, there are a plurality of files named “SIDE.ifo”(“SIDE0.ifo”, “SIDE1.ifo”, . . . , and “SIDEk.ifo”). The files“SIDE.ifo” are loaded with side information relating to a plurality ofcontent programs, that is, audio-visual (AV) programs.

In addition, under the folder “LIB”, there are folders “PR”, “PR1”, . .. , and “PRn” loaded with information pieces “PR0.dat”, “PR1.dat”, and“PRn.dat”, respectively. The information pieces “PR0.dat”, “PR1.dat”,and “PRn.dat” are designed for link with AV multiplexing-resultant bitstreams.

Furthermore, under the folder “LIB”, there are folders “PL0”, “PL1”, . .. and “PLn” for containing after-recorded audio files (files loaded withafter-recorded audio information). For example, in the case where “m”after-recorded audio files relating to the folder “PR0” are made andrecorded, list information pieces “AF0-1.dat”, “AF0-2.dat”, . . . ,“AF0-m.dat” are placed in the folder “PL0” as information for link withthe after-recorded audio files.

As understood from the above description, link information relating toAV multiplexing-resultant files and after-recorded audio files isrecorded on the recording medium 108. Desired content information can bereproduced from the recording medium on the basis of the linkinformation.

With reference to FIG. 9, each file “SIDE.ifo” is in a format having ahierarchical structure. Specifically, each file “SIDE.ifo” has afirst-level segment “TOTAL_MANAGER_IFO” containing second-level segments“GENERAL_IFO” and “CNTNT_IFO”. The second-level segment “GENERAL_IFO” isloaded with parameters relating to the whole of the present sideinformation.

The second-level segment “GENERAL_IFO” is of a syntax structure shown inFIG. 10. Specifically, the second-level segment “GENERAL_IFO” includesinformation pieces having syntax names “System_id”, “TMG_IFO_length”,“Num of PR_IFO”, “Num of PL_IFO”, “Start Address of PR_IFO”, and “StartAddress of PL-IFO”, respectively. The information piece “System_id” is a32-bit signal representing the type of the present informationstructure. The information piece “TMG_IFO_length” is a 32-bit signalrepresenting the whole manager length. The information piece “Num ofPR_IFO” is an 8-bit signal representing the number of programinformation pieces “PR_IFO” which will be explained later. Theinformation piece “Num of PL_IFO” is an 8-bit signal representing thenumber of after-recorded information pieces “PL_IFO” which will beexplained later. The information piece “Start Address of PR_IFO” is a32-bit signal representing the head address of a first programinformation piece “PR_IFO_0”. The information piece “Start Address ofPL_IFO” is a 32-bit signal representing the head address of a firstafter-recorded information piece “PL_IFO_0”.

The second-level segment “CNTNT_IFO” in FIG. 9 contains third-levelsegments “PR_IFO_0”, “PR_IFO_1”, . . . , and “PR_IFO_m” loaded withinformation pieces which relate to content programs (or AVmultiplexing-resultant files) respectively. Furthermore, thesecond-level segment “CNTNT_IFO” contains third-level segments“PL_IFO_0”, “PL_IFO_1”, . . . , and “PL_IFO_n” loaded with informationpieces for after-recorded audio data which relate to the contentprograms (or the AV multiplexing-resultant files) respectively. Forexample, in the case where after-recorded audio data corresponding tothe third-level segment “PR_IFO_0” are present, the third-level segment“PL-IFO_0” contains a fourth-level segment “PLAYL_IFO” loaded withinformation (play list information) relating to the after-recorded audiofile.

The third-level segments “PR_IFO₁₃ 0”, “PR_IFO_1”, . . . , and“PR_IFO_m” are similar in structure. Only the third-level segment“PR_IFO_0” will be explained in more detail. As shown in FIG. 11, thethird-level segment “PR_IFO_0” contains fourth-level segments “PROG_IFO”and “IDX_IFO”. The fourth-level segment “PROG_IFO” is loaded withcontent-program-related information. The fourth-level segment “IDX_IFO”contains fifth-level segments “IDX_IFO_0”, “IDX_IFO_1”, . . . , and“IDX_IFO_n” loaded with information pieces which relate to respectiveindexes of the related audio-visual program. The fifth-level segments“IDX_IFO_0”, “IDX_IFO_1”, . . . , and “IDX_IFO_n” are similar instructure. For example, the fifth-level segment “IDX_IFO_0” has asixth-level segment “INDEX_IFO” in which a portion of the relatedaudio-visual program can be registered as index information.

The fourth-level segment “PROG_IFO” is of a syntax structure shown inFIG. 12. Specifically, each fourth-level segment “PROG_IFO” includesinformation pieces having syntax names “Size of PROG_IFO”, “PR number”,“Content type”, and “Component type”, respectively. The informationpiece “Size of PROG_IFO” is a 32-bit signal representing the size of thepresent fourth-level segment “PROG_IFO”. The information piece “PRnumber” is an 8-bit signal representing the designation number (theidentification number) assigned to the related audio-visual program. ThePR number is variable among “0”-“255” corresponding to differentaudio-visual programs respectively. The information piece “Content type”is an 8-bit signal representing the type of the related audio-visualprogram. The information piece “Component type” is an 8-bit signalrepresenting the type of related data, that is, representing whetherrelated data are of video, audio, or other.

As previously mentioned, in the presence of an after-recorded audiofile, play list information “PLAYL_IFO” is provided. In the presence of“m” after-recorded audio files corresponding to the folder “PR0”, listinformation pieces “AF0-1.dat”, “AF0-2.dat”, . . . , “AF0-m.dat” areplaced in the folder “PL0” as information for link with theafter-recorded audio files.

The play list information “PLAYL_IFO” is of a syntax structure shown inFIG. 13. Specifically, the play list information “PLAYL_IFO” includesinformation pieces having syntax names “PR_number” and “AF₁₃ number”respectively. The information piece “PR_number” is an 8-bit signalrepresenting the designation number (the identification number) assignedto the related main data, that is, the related audio-visual program. Theinformation piece “AF_number” is an 8-bit signal representing thedesignation number (the identification number) assigned to the relatedafter-recorded audio data.

Regarding each of “n” AV multiplexing-resultant streams “PRj.dat (j=0,1, . . . , n)” in the folders “PRj” of FIG. 8, “m” after-recorded audiodata can be registered in accordance with user's request. When thenumber “m” is equal to “0”, an AV multiplexing-resultant file is usedwithout employing after-recorded audio data. When the number “m” isequal to “1” or greater, the recording of after-recording-purpose audiodata is permitted. In this case, at least one after-recorded audio fileis made and recorded. Also, the after-recorded audio data are reproducedor transmitted.

The index information “INDEX_IFO” in FIG. 11 is of a syntax structure inFIG. 14. Specifically, the index information “INDEX_IFO” includesinformation pieces having syntax names “INDEX number”, “Playback Time”,“Start Address”, and “End Address”, respectively. The information piece“INDEX number” is an 8-bit signal representing the serial numberassigned to the related index. The information piece “Playback Time” isa 40-bit signal representing the playback time of the related index. Theinformation piece “Start Address” is a 64-bit signal representing theaddress of the starting point of the related index. The informationpiece “End Address” is a 64-bit signal representing the address of theending point of the related index.

FIG. 15 shows the format of a TS recorded on a hard disk whichconstitutes the recording medium 108. As shown in FIG. 15, the TS isformed by a sequence of TS units. Each TS unit is composed of packets.Each packet is composed of a 25-bit time stamp and a 188-byte MPEG TSpacket.

FIG. 16 shows a reproducing apparatus using the MPEG-system decodingapparatus of FIG. 3. As shown in FIG. 16, the reproducing apparatusincludes a user interface 120 connected with a CPU 121. The userinterface 120 can be handled by a user. The CPU 121 has a combination ofan input/output port, a processing section, a ROM, and a RAM. The CPU121 operates in accordance with a control program stored in the ROM orthe RAM. The control program is designed to enable the CPU 121 toimplement operation steps mentioned later. By handling the userinterface 120, operation of the reproducing apparatus can be changedamong different modes including a first mode and a second mode. Thefirst operation mode is designed to reproduce main data(multiplexing-resultant data). The second operation mode is designed toreproduce after-recorded audio data instead of audio information in maindata while synchronously reproducing video information in the main data.When the user interface 120 is handled, an operation-mode designationsignal is inputted therefrom into the CPU 121. The operation-modedesignation signal indicates which of the first operation mode and thesecond operation mode is desired. The CPU 121 transfers theoperation-mode designation signal to a signal selector 114. Also,identification (ID) information can be inputted into the CPU 121 byhandling the user interface 120. The CPU 121 transfers theidentification information to an identification information detector123.

When the operation-mode designation signal indicates that the firstoperation mode is desired, that is, when the reproduction of main datais desired, the reproducing apparatus operates as follows. A signal foridentifying desired main data is inputted into the CPU 121 by handlingthe user interface 120. The identifying signal represents thedesignation number (the identification number) assigned to the desiredmain data. The CPU 121 transfers the identifying signal to theidentification information detector 123. The identification informationdetector 123 derives the identification number of the desired main datafrom the identifying signal. The identification information detector 123notifies the reading controller 111 of the identification number of thedesired main data. The identification information detector 123 ordersthe reading controller 111 to read out, from a recording medium 108, amain-data file having a name corresponding to the identification numberof the desired main data. Thus, the reading controller 111 implementsthe read-out of the desired main data from the recording medium 108. Inthis way, the reading controller 111 reads out desired main data, thatis, desired multiplexing-resultant data, from the recording medium 108.The read-out main data are sent from the reading controller 111 to abuffer 112 a. The main data are stored in the buffer 112 a before beingoutputted therefrom to a demultiplexer 113. The demultiplexer 113separates the main data into video data and audio data. The video dataare sent from the demultiplexer 113 to a time stamp detector 115 a. Theaudio data are sent from the demultiplexer 113 to the signal selector114. The signal selector 114 selects the audio data from thedemultiplexer 113 in response to the operation-mode designation signal,and passes the selected audio data to a time stamp detector 115 b.

The time stamp detector 115 a detects timing information (every PCR,every video PTS, and every video DTS) in the video data. The time stampdetector 115 a sends the detected PCR, the detected video PTS, and thedetected video DTS to a time stamp comparator 124. The time stampdetector 115 a passes the video data to a PES de-packeting device 116 a.The time stamp detector 115 b detects timing information (every PCR,every audio PTS, and every audio DTS) in the audio data. The time stampdetector 115 b sends the detected PCR, the detected audio PTS, and thedetected audio DTS to the time stamp comparator 124. The time stampdetector 115 b passes the audio data to a PES de-packeting device 116 b.

The PES de-packeting device 116 a de-packets the video data (a sequenceof PES packets) to generate de-packeting-resultant video data. The PESde-packeting device 116 a outputs the depacketing-resultant video datato a video decoder 117 a. The PES de-packeting device 116 b de-packetsthe audio data (a sequence of PES packets) to generatede-packeting-resultant audio data. The PES de-packeting device 116 boutputs the de-packeting-resultant audio data to an audio decoder 117 b.The video decoder 117 a implements the MPEG decoding of the video datato generate decoding-resultant video data. The video decoder 117 astores the decoding-resultant video data into a memory 118 a. The audiodecoder 117 b implements the MPEG decoding of the audio data to generatedecoding-resultant audio data. The audio decoder 117 b stores thedecoding-resultant audio data into a memory 118 b.

A signal generator 124 a outputs a 27-MHz clock signal to the time stampcomparator 124. On the basis of the 27-MHz clock signal, the time stampcomparator 124 generates a reference clock signal denoting referenceclock time. The time stamp comparator 124 locks the frequency of thereference clock signal to a frequency indicated by the PCR. Therefore,the reference clock time is decided by the PCR. The time stampcomparator 124 compares time denoted by the video DTS with the referenceclock time to decide whether or not the video DTS time agrees with thereference clock time. When the video DTS time agrees with the referenceclock time, the time stamp comparator 124 enables the video decoder 117a to start the decoding of the video data originating from the PESpacket having the related video DTS. The time stamp comparator 124compares time denoted by the audio DTS with the reference clock time todecide whether or not the audio DTS time agrees with the reference clocktime. When the audio DTS time agrees with the reference clock time, thetime stamp comparator 124 enables the audio decoder 117 b to start thedecoding of the audio data originating from the PES packet having therelated audio DTS.

The time stamp comparator 124 compares time denoted by the video PTSwith the reference clock time to decide whether or not the video PTStime agrees with the reference clock time. When the video PTS timeagrees with the reference clock time, the time stamp comparator 124enables the memory 118 a to output the decoding-resultant video data toa display monitor 119 a for an indication purpose. The time stampcomparator 124 compares time denoted by the audio PTS with the referenceclock time to decide whether or not the audio PTS time agrees with thereference clock time. When the audio PTS time agrees with the referenceclock time, the time stamp comparator 124 enables the memory 118 b tooutput the decoding-resultant audio data to loudspeakers 119 b for aplayback purpose. The video PTS and the audio PTS are preset so that thedecoding-resultant video data and the decoding-resultant audio data canbe synchronously played back.

When the operation-mode designation signal indicates that the secondoperation mode is desired, that is, when the reproduction ofafter-recorded audio data and main-data video information is desired,the reproducing apparatus operates as follows. Signals for identifyingdesired main data and desired after-recorded audio data are inputtedinto the CPU 121 by handling the user interface 120. The identifyingsignals correspond to information pieces “PR_number” and “AF_number” inFIG. 13 which represent the designation numbers (the identificationnumbers) assigned to the desired main data and the desiredafter-recorded audio data. The CPU 121 transfers the identifying signalsto the identification information detector 123. The reading controller111 reads out play list information “PLAYL_IFO” (see FIG. 13) from arecording medium 108, and sends the play list information “PLAYL_IFO” tothe identification information detector 123. The identificationinformation detector 123 detects the identification numbers of thedesired main data and the desired after-recorded audio data in responseto the identifying signals by referring to the play list information“PLAYL_IFO”. The identification information detector 123 notifies thereading controller 111 of the identification numbers of the desired maindata and the desired after-recorded audio data. The identificationinformation detector 123 orders the reading controller 111 toalternately read out, from the recording medium 108, a main-data fileand an after-recorded audio file having names corresponding to theidentification numbers of the desired main data and the desiredafter-recorded audio data. Thus, the reading controller 111 implementsthe read-out of the desired main data and the desired after-recordedaudio data from the recording medium 108 on an alternate time-sharingburst basis. The read-out main data are sent from the reading controller111 to the buffer 112 a. The main data are stored in the buffer 112 abefore being outputted therefrom to the demultiplexer 113. Thedemultiplexer 113 separates the main data into video data and audiodata. The video data are sent from the demultiplexer 113 to the timestamp detector 115 a. The audio data are sent from the demultiplexer 113to the signal selector 114. The read-out after-recorded audio data aresent from the reading controller 111 to the buffer 112 b. Theafter-recorded audio data are stored in the buffer 112 b before beingoutputted therefrom to the signal selector 114. The signal selector 114selects the after-recorded audio data from the buffer 112 b in responseto the operation-mode designation signal, and passes the selectedafter-recorded audio data to the time stamp detector 115 b. In otherwords, the signal selector 114 rejects the audio data from thedemultiplexer 113.

The time stamp detector 115 a detects timing information (every PCR,every video PFS, and every video DTS) in the video data. The time stampdetector 115 a sends the detected PCR, the detected video PTS, and thedetected video DTS to the time stamp comparator 124. The time stampdetector 115 a passes the video data to the PES de-packeting device 116a. The time stamp detector 115 b detects timing information (every PCR,every audio PTS, and every audio DTS) in the after-recorded audio data.The time stamp detector 115 b sends the detected PCR, the detected audioPTS, and the detected audio DTS to the time stamp comparator 124. Thetime stamp detector 115 b passes the after-recorded audio data to thePES de-packeting device 116 b.

The PES de-packeting device 116 a de-packets the video data (a sequenceof PES packets) to generate de-packeting-resultant video data. The PESde-packeting device 116 a outputs the depacketing-resultant video datato the video decoder 117 a. The PES de-packeting device 116 b de-packetsthe after-recorded audio data (a sequence of PES packets) to generatede-packeting-resultant audio data. The PES de-packeting device 116 boutputs the depacketing-resultant audio data to the audio decoder 117 b.The video decoder 117 a implements the MPEG decoding of the video datato generate decoding-resultant video data. The video decoder 117 astores the decoding-resultant video data into the memory 118 a. Theaudio decoder 117 b implements the MPEG decoding of the audio data togenerate decoding-resultant audio data. The audio decoder 117 b storesthe decoding-resultant audio data into the memory 118 b.

The signal generator 124 a outputs the 27-MHz clock signal to the timestamp comparator 124. On the basis of the 27-MHz clock signal, the timestamp comparator 124 generates a reference clock signal denotingreference clock time. The time stamp comparator 124 locks the frequencyof the reference clock signal to a frequency indicated by the PCR.Therefore, the reference clock time is decided by the PCR. The timestamp comparator 124 compares time denoted by the video DTS with thereference clock time to decide whether or not the video DTS time agreeswith the reference clock time. When the video DTS time agrees with thereference clock time, the time stamp comparator 124 enables the videodecoder 117 a to start the decoding of the video data originating fromthe PES packet having the related video DTS. The time stamp comparator124 compares time denoted by the audio DTS with the reference clock timeto decide whether or not the audio DTS time agrees with the referenceclock time. When the audio DTS time agrees with the reference clocktime, the time stamp comparator 124 enables the audio decoder 117 b tostart the decoding of the audio data originating from the PES packethaving the related audio DTS.

The time stamp comparator 124 compares time denoted by the video PTSwith the reference clock time to decide whether or not the video PTStime agrees with the reference clock time. When the video PTS timeagrees with the reference clock time, the time stamp comparator 124enables the memory 118 a to output the decoding-resultant video data tothe display monitor 119 a for an indication purpose. The time stampcomparator 124 compares time denoted by the audio PTS with the referenceclock time to decide whether or not the audio PTS time agrees with thereference clock time. When the audio PTS time agrees with the referenceclock time, the time stamp comparator 124 enables the memory 118 b tooutput the decoding-resultant audio data to the loudspeakers 119 b for aplayback purpose. The video PTS and the audio PTS are preset so that thedecoding-resultant video data and the decoding-resultant audio data (theafter-recorded audio data) can be synchronously played back.

As previously mentioned, the CPU 121 is connected with the userinterface 120. The CPU 121 can receive an operation-mode designationsignal from the user interface 120. Also, the CPU 121 can receiveidentification (ID) information from the user interface 120. The CPU 121responds to the operation-mode designation signal and the IDinformation. The CPU 121 is connected with the devices 111, 112 a, 112b, 113, 114, 115 a, 115 b, 116 a, 116 b, 117 a, 117 b, 118 a, 118 b,123, and 124. As previously mentioned, the CPU 121 has a combination ofan input/output port, a processing section, a ROM, and a RAM. The CPU121 operates in accordance with a control program stored in the ROM orthe RAM. Preferably, the control program is transmitted into the CPU 121from a recording medium. The control program may be downloaded into theCPU 121 via a communication network.

FIG. 19 is a flowchart of a segment of the control program for the CPU121. As shown in FIG. 19, a first step S30 controls the identificationinformation detector 123 and the reading controller 111 to accessdesired main data (desired multiplexing-resultant data) or both desiredmain data and desired after-recorded audio data on a recording medium108. After the step S30, the program advances to a step S31.

The step S31 controls the reading controller 111 to read out the desiredmain data or both the desired main data and the desired after-recordedaudio data from the recording medium 108. The step S31 controls thereading controller 111 and the buffer 112 a to store the read-out maindata (the read-out multiplexing-resultant data) in the buffer 112 a. Inthe presence of the desired after-recorded audio data, the step S31controls the reading controller 111 and the buffer 112 b to store theread-out after-recorded audio data in the buffer 112 b.

A step S32 following the step S31 monitors the amount (the number ofbits) of the multiplexing-resultant data in the buffer 112 a. The stepS32 decides whether or not the amount of the multiplexing-resultant datain the buffer 112 a exceeds a first prescribed amount. In addition, thestep S32 monitors the amount (the number of bits) of the after-recordedaudio data in the buffer 112 b. The step S32 decides whether or not theamount of the after-recorded audio data in the buffer 112 b exceeds asecond prescribed amount. In the case where the amount of themultiplexing-resultant data in the buffer 112 a exceeds the firstprescribed amount and also the amount of the after-recorded audio datain the buffer 112 b exceeds the second prescribed amount, the programadvances from the step S32 to a step S33. Otherwise, the program returnsfrom the step S32 to the step S31.

The step S33 decides whether or not the after-recorded-data reproducingoperation mode (the second operation mode) is desired by referring to anoperation-mode designation signal outputted from the user interface 109.When the after-recorded data reproducing operation mode is desired, theprogram advances from the step S33 to a step S34. Otherwise, the programadvances from the step S33 to a step S35.

The step S35 controls the buffer 112 a to output the main data (themultiplexing-resultant data) to the demultiplexer 113. The step S35controls the demultiplexer 113 to separate the main data into video dataand audio data. The video data are sent from the demultiplexer 113 tothe time stamp detector 115 a. The audio data are sent from thedemultiplexer 113 to the signal selector 114. The step S35 controls thesignal selector 114 in response to the operation-mode designation signalso that the signal selector 114 selects the audio data from thedemultiplexer 113 and passes the selected audio data to the time stampdetector 115 b.

A step S36 following the step S35 controls the time stamp detector 115 ato detect timing information (every PCR, every video PTS, and everyvideo DTS) in the video data. The time stamp detector 115 a sends thedetected PCR, the detected video PTS, and the detected video DTS to thetime stamp comparator 124. The time stamp detector 115 a passes thevideo data to the PES de-packeting device 116 a. The step S36 controlsthe time stamp detector 115 b to detect timing information (every PCR,every audio PTS, and every audio DTS) in the audio data. The time stampdetector 115 b sends the detected PCR, the detected audio PTS, and thedetected audio DTS to the time stamp comparator 124. The time stampdetector 115 b passes the audio data to the PES depacketing device 116b.

A step S37 subsequent to the step S36 controls the PES depacketingdevice 116 a to de-packet the video data (a sequence of PES packets) togenerate de-packeting-resultant video data. The PES de-packeting device116 a outputs the de-packeting-resultant video data to the video decoder117 a. The step S37 controls the PES de-packeting device 116 b tode-packet the audio data (a sequence of PES packets) to generatede-packeting-resultant audio data. The PES de-packeting device 116 boutputs the de-packeting-resultant audio data to the audio decoder 117b.

A step S38 following the step S37 controls the video decoder 117 a toimplement the MPEG decoding of the video data to generatedecoding-resultant video data. The video decoder 117 a stores thedecoding-resultant video data into the memory 118 a. The step S38controls the audio decoder 117 b to implement the MPEG decoding of theaudio data to generate decoding-resultant audio data. The audio decoder117 b stores the decoding-resultant audio data into the memory 118 b.

A step S39 subsequent to the step S38 implements synchronous playback ofthe decoding-resultant video data and the decoding-resultant audio data.Specifically, the step S39 controls the time stamp comparator 124 todecide reference clock time in response to the PCR. The step S39controls the time stamp comparator 124 to compare time denoted by thevideo DTS with the reference clock time, and thereby to decide whetheror not the video DTS time agrees with the reference clock time. When thevideo DTS time agrees with the reference clock time, the time stampcomparator 124 enables the video decoder 117 a to start the decoding ofthe video data originating from the PES packet having the related videoDTS. The step S39 controls the time stamp comparator 124 to compare timedenoted by the audio DTS with the reference clock time, and thereby todecide whether or not the audio DTS time agrees with the reference clocktime. When the audio DTS time agrees with the reference clock time, thetime stamp comparator 124 enables the audio decoder 117 b to start thedecoding of the audio data originating from the PES packet having therelated audio DTS.

The step S39 controls the time stamp comparator 124 to compare timedenoted by the video PTS with the reference clock time, and thereby todecide whether or not the video PTS time agrees with the reference clocktime. When the video PTS time agrees with the reference clock time, thetime stamp comparator 124 enables the memory 118 a to output thedecoding-resultant video data to the display monitor 119 a for anindication purpose. The step S39 controls the time stamp comparator 124to compare time denoted by the audio PTS with the reference clock time,and thereby to decide whether or not the audio PTS time agrees with thereference clock time. When the audio PTS time agrees with the referenceclock time, the time stamp comparator 124 enables the memory 118 b tooutput the decoding-resultant audio data to the loudspeakers 119 b for aplayback purpose. The video PTS and the audio PTS are preset so that thedecoding-resultant video data and the decoding-resultant audio data canbe synchronously played back. After the step S39, the program advancesto a step S40.

The step S34 controls the buffer 112 a to output the main data (themultiplexing-resultant data) to the demultiplexer 113. The step S34controls the demultiplexer 113 to separate the main data into video dataand audio data. The video data are sent from the demultiplexer 113 tothe time stamp detector 115 a. The audio data are sent from thedemultiplexer 113 to the signal selector 114. The step S34 controls thebuffer 112 b to output the after-recorded audio data to the signalselector 114. The step S34 controls the signal selector 114 in responseto the operation-mode designation signal so that the signal selector 114selects the after-recorded audio data from the buffer 112 b and passesthe selected after-recorded audio data to the time stamp detector 115 b.

A step S41 following the step S34 controls the time stamp detector 115 ato detect timing information (every PCR, every video PTS, and everyvideo DTS) in the video data. The time stamp detector 115 a sends thedetected PCR, the detected video PTS, and the detected video DTS to thetime stamp comparator 124. The time stamp detector 115 a passes thevideo data to the PES de-packeting device 116 a. The step S41 controlsthe time stamp detector 115 b to detect timing information (every PCR,every audio PTS, and every audio DTS) in the after-recorded audio data.The time stamp detector 115 b sends the detected PCR, the detected audioPTS, and the detected audio DTS to the time stamp 25 comparator 124. Thetime stamp detector 115 b passes the after-recorded audio data to thePES de-packeting device 116 b.

A step S42 subsequent to the step S41 controls the PES depacketingdevice 116 a to de-packet the video data (a sequence of PES packets) togenerate de-packeting-resultant video data. The PES de-packeting device116 a outputs the de-packeting-resultant video data to the video decoder117 a. The step S42 controls the PES de-packeting device 116 b tode-packet the after-recorded audio data (a sequence of PES packets) togenerate de-packeting-resultant audio data. The PES de-packeting device116 b outputs the de-packeting-resultant audio data to the audio decoder117 b.

A step S43 following the step S42 controls the video decoder 117 a toimplement the MPEG decoding of the video data to generatedecoding-resultant video data. The video decoder 117 a stores thedecoding-resultant video data into the memory 118 a. The step S43controls the audio decoder 117 b to implement the MPEG decoding of theaudio data to generate decoding-resultant audio data. The audio decoder117 b stores the decoding-resultant audio data into the memory 118 b.

A step S44 subsequent to the step S43 implements synchronous playback ofthe decoding-resultant video data and the decoding-resultant audio data(the after-recorded audio data). Specifically, the step S44 controls thetime stamp comparator 124 to decide reference clock time in response tothe PCR. The step S44 controls the time stamp comparator 124 to comparetime denoted by the video DTS with the reference clock time, and therebyto decide whether or not the video DTS time agrees with the referenceclock time. When the video DTS time agrees with the reference clocktime, the time stamp comparator 124 enables the video decoder 117 a tostart the decoding of the video data originating from the PES packethaving the related video DTS. The step S44 controls the time stampcomparator 124 to compare time denoted by the audio DTS with thereference clock time, and thereby to decide whether or not the audio DTStime agrees with the reference clock time. When the audio DTS timeagrees with the reference clock time, the time stamp comparator 124enables the audio decoder 117 b to start the decoding of the audio dataoriginating from the PES packet having the related audio DTS.

The step S44 controls the time stamp comparator 124 to compare timedenoted by the video PTS with the reference clock time, and thereby todecide whether or not the video PTS time agrees with the reference clocktime. When the video PTS time agrees with the reference clock time, thetime stamp comparator 124 enables the memory 118 a to output thedecoding-resultant video data to the display monitor 119 a for anindication purpose. The step S44 controls the time stamp comparator 124to compare time denoted by the audio PTS with the reference clock time,and thereby to decide whether or not the audio PTS time agrees with thereference clock time. When the audio PTS time agrees with the referenceclock time, the time stamp comparator 124 enables the memory 118 b tooutput the decoding-resultant audio data to the loudspeakers 119 b for aplayback purpose. The video PTS and the audio PTS are preset so that thedecoding-resultant video data and the decoding-resultant audio data (theafter-recorded audio data) can be synchronously played back. After thestep S44, the program advances to the step S40.

The step S40 decides whether or not remaining desired data to be playedback exist. When remaining desired data exist, the program returns fromthe step S40 to the step S30. Otherwise, the program advances from thestep S40, and then the current execution cycle of the program segmentends.

FIG. 17 shows a transmission apparatus. As shown in FIG. 17, thetransmission apparatus includes a user interface 120A connected with aCPU 121A. The user interface 120A can be handled by a user. The CPU 121Ahas a combination of an input/output port, a processing section, a ROM,and a RAM. The CPU 121A operates in accordance with a control programstored in the ROM or the RAM. The control program is designed to enablethe CPU 121A to implement operation steps mentioned later. By handlingthe user interface 120A, operation of the transmission apparatus can bechanged among different modes including a first mode and a second mode.The first operation mode is designed to transmit main data(multiplexing-resultant data). The second operation mode is designed totransmit after-recorded audio data instead of audio information in maindata while transmitting video information in the main data. When theuser interface 120A is handled, an operation-mode designation signal isinputted therefrom into the CPU 121A. The operation-mode designationsignal indicates which of the first operation mode and the secondoperation mode is desired. The CPU 121A transfers the operation-modedesignation signal to a signal selector 125. Also, identification (ID)information can be inputted into the CPU 121A by handling the userinterface 120A. The CPU 121A transfers the identification information toan identification information detector 123.

When the operation-mode designation signal indicates that the firstoperation mode is desired, that is, when the transmission of main datais desired, the transmission apparatus operates as follows. A readingcontroller 111 reads out main data, that is, multiplexing-resultantdata, from a recording medium 108. The read-out main data are sent fromthe reading controller 111 to a buffer 112 a. The main data are storedin the buffer 112 a before being outputted therefrom to the signalselector 125. The signal selector 125 selects the main data from thebuffer 112 a, and passes the main data to a buffer 126. The main dataare stored in the buffer 126 before being outputted therefrom to atransmission line.

When the operation-mode designation signal indicates that the secondoperation mode is desired, that is, when the transmission ofafter-recorded audio data and main-data video information is desired,the transmission apparatus operates as follows. Signals for identifyingdesired main data and desired after-recorded audio data are inputtedinto the CPU 121A by handling the user interface 120A. The identifyingsignals correspond to information pieces “PR_number” and “AF_number” inFIG. 13 which represent the designation numbers (the identificationnumbers) assigned to the desired main data and the desiredafter-recorded audio data. The CPU 121A transfers the identifyingsignals to the identification information detector 123. The readingcontroller 111 reads out play list information “PLAYL_IFO” from arecording medium 108. The reading controller 111 sends the play listinformation “PLAYL_IFO” to the identification information detector 123.The identification information detector 123 detects the identificationnumbers of the desired main data and the desired after-recorded audiodata in response to the identifying signals by referring to the playlist information “PLAYL_IFO” (see FIG. 13). The identificationinformation detector 123 notifies the reading controller 111 of theidentification numbers of the desired main data and the desiredafter-recorded audio data. The identification information detector 123orders the reading controller 111 to alternately read out, from therecording medium 108, a main-data file and an after-recorded audio filehaving names corresponding to the identification numbers of the desiredmain data and the desired after-recorded audio data. Thus, the readingcontroller 111 implements the read-out of the desired main data and thedesired after-recorded audio data from the recording medium 108 on analternate time-sharing burst basis. The read-out main data are sent fromthe reading controller 111 to the buffer 112 a. The main data are storedin the buffer 112 a before being outputted therefrom to the signalselector 125. The read-out after-recorded audio data are sent from thereading controller 111 to the buffer 112 b. The after-recorded audiodata are stored in the buffer 112 b before being outputted therefrom tothe signal selector 125. The signal selector 125 replaces audio data inthe main data with the after-recorded audio data from the buffer 112 bin response to the operation-mode designation signal, and therebyconverts the original main data into new main data. The new main dataare sent from the signal selector 125 to the buffer 126. The new maindata are stored in the buffer 126 before being outputted therefrom tothe transmission line. During the conversion of the original main datainto the new main data, it is unnecessary to alter every PCR, every DTS,and every PTS. Preferably, the signal selector 125 includes acalculator. The calculator may be formed by the CPU 121A. For packets ofthe new main data which have been subjected to audio-data replacement,the calculator computes new CRC (cyclic redundancy check) code words andreplaces old CRC code words with the new ones.

As understood from the previous description, regarding the generation ofmain data, it is supposed that a data piece related to the type ofafter-recording-purpose audio data is previously produced in elementdata, and that the transmission rate of the after-recording-purposeaudio data is equal to that of original audio data in the main data.

Second Embodiment

A second embodiment of this invention is similar to the first embodimentthereof except for design changes mentioned later. According to thefirst embodiment of this invention, regarding the generation of maindata, it is supposed that a data piece related to the type ofafter-recording-purpose audio data is previously produced in elementdata, and that the transmission rate of the after-recording-purposeaudio data is equal to that of original audio data in the main data.According to the second embodiment of this invention, regarding thegeneration of main data, element data equal in transmission rate toexpected after-recording-purpose audio data are previously recorded on amultiplexed basis as dummy data.

Third Embodiment

A third embodiment of this invention is similar to the first embodimentthereof except for design changes mentioned later. According to thethird embodiment of this invention, in the case whereafter-recording-purpose audio data are expected to be recorded, elementdata equal in transmission rate to the expected after-recording-purposeaudio data are previously recorded on a multiplexed basis as dummy data.When actual after-recording-purpose audio data are generated, thepreviously-recorded dummy data are replaced with the actualafter-recording-purpose audio data. During the replacement of thepreviously-recorded dummy data with the actual after-recording-purposeaudio data, it is unnecessary to alter every PCR, every DTS, and everyPTS.

In the case where the transmission rate of actualafter-recording-purpose audio data differs from the expectedtransmission rate, video data and the actual after-recording-purposeaudio data are recorded on a multiplexed basis. In this case, it isnecessary to renew every PCR, every DTS, and every PTS.

Advantages Provided by Embodiments

The first, second, and third embodiments of this invention provide thefollowing advantages.

An audio signal and a video signal are compressively encoded intoencoding-resultant audio data and encoding-resultant video data. Anaudio time stamp for synchronous reproduction is recorded in every unitof the encoding-resultant audio data while a video time stamp forsynchronous reproduction is recorded in every unit of theencoding-resultant video data. The encoding-resultant audio data and theencoding-resultant video data are multiplexed into main data. The maindata are recorded on a recording medium. One or more types ofafter-recording-purpose data for either the encoding-resultant audiodata or the encoding-resultant video data are also recorded on therecording medium separately from the recorded main data. Thus, theafter-recording-purpose data are recorded without being multiplexed.Accordingly, it is possible to provide a format for easily recordingafter-recording-purpose data. Furthermore, it is possible to record manytypes of after-recording-purpose data.

An audio signal and a video signal are compressively encoded intoencoding-resultant audio data and encoding-resultant video data. Anaudio time stamp for synchronous reproduction is recorded in every unitof the encoding-resultant audio data while a video time stamp forsynchronous reproduction is recorded in every unit of theencoding-resultant video data. The encoding-resultant audio data and theencoding-resultant video data are multiplexed into main data. The maindata are recorded on a recording medium. One or more types ofafter-recording-purpose data for either the encoding-resultant audiodata or the encoding-resultant video data are also recorded on therecording medium separately from the recorded main data. Thus, theafter-recording-purpose data are recorded without being multiplexed. Theafter-recording-purpose data are read out from the recording medium on atime sharing basis before being reproduced. Accordingly, it is possibleto easily identify the types of the after-recording-purpose data.Furthermore, it is possible to reproduce the after-recording-purposedata without multiplexing them again.

An audio signal and a video signal are compressively encoded intoencoding-resultant audio data and encoding-resultant video data. Anaudio time stamp for synchronous reproduction is recorded in every unitof the encoding-resultant audio data while a video time stamp forsynchronous reproduction is recorded in every unit of theencoding-resultant video data. The encoding-resultant audio data and theencoding-resultant video data are multiplexed into main data. The maindata are recorded on a recording medium. One or more types ofafter-recording-purpose data for either the encoding-resultant audiodata or the encoding-resultant video data are also recorded on therecording medium separately from the recorded main data. Thus, theafter-recording-purpose data are recorded without being multiplexed. Themain data and the after-recording-purpose data are read out from therecording medium on a time sharing basis. The after-recording-purposedata replace corresponding element data in the main data so that themain data are converted into new main data. The new main data aretransmitted. Accordingly, it is possible to transmit data in conformitywith the MEG standards for multiplexing. An MPEG reproducer in atransmission destination can reproduce the transmitted data containingthe after-recording-purpose data.

In the case where after-recording-purpose data are recorded on arecording medium, recorded main data on the recording medium remainunchanged. Accordingly, it is possible to reproduce the main data asthey are. In the case where after-recording-purpose data are reproducedfrom a recording medium before being transmitted, recorded main data onthe recording medium remain unchanged. Accordingly, it is possible toreproduce the main data as they are. In the case where a plurality oftypes of after-recording-purpose data are recorded on a recordingmedium, a user can selectively enjoy one or more of the types ofafter-recording-purpose data.

There are identification information pieces for identifying a pluralityof types of after-recording-purpose data, respectively. Time stamps forreproduction synchronous with main data, and the identificationinformation pieces are added to the plurality of types ofafter-recording-purpose data. Each of the types ofafter-recording-purpose data is made into a bit stream without beingmultiplexed with the main data. Accordingly, the main data remainunchanged. Thus, even in the case where main data andafter-recording-purpose data are recorded on a recording medium, it ispossible to reproduce the main data from the recording medium as theyare. In the case where main data and a plurality of types ofafter-recording-purpose data are recorded on a recording medium, it ispossible to selectively reproduce one of the types ofafter-recording-purpose data instead of the main data to convert themain data into new main data.

1. An after-recording apparatus comprising: first means forcompressively encoding an audio signal into encoding-resultant audiodata divided in units; second means for compressively encoding a videosignal into encoding-resultant video data divided in units; third meansfor adding an audio time stamp for audio-video synchronous reproductionto every unit of the encoding-resultant audio data generated by thefirst means to generate time-stamp-added audio data; fourth means foradding a video time stamp for audio-video synchronous reproduction toevery unit of the encoding-resultant video data generated by the secondmeans to generate time-stamp-added video data; fifth means formultiplexing the time-stamp-added audio data generated by the thirdmeans and the time-stamp-added video data generated by the fourth meansinto main data; sixth means for, to first after-recording-purposeencoding-resultant data for at least one of (1) the encoding-resultantaudio data generated by the first means and (2) the encoding-resultantvideo data generated by the second means which form the main data,adding a time stamp for reproduction synchronous with a portion of themain data and identification information for identifying the firstafter-recording-purpose encoding-resultant data to convert the firstafter-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and seventh means forrecording the main data, the second after-recording-purposeencoding-resultant data, and play list information on a recordingmedium, the play list information providing link between the main dataand the second after-recording-purpose encoding-resultant data.
 2. Acomputer-readable medium storing a computer program for after recording,the computer program comprising the steps of: compressively encoding anaudio signal into encoding-resultant audio data divided in units;compressively encoding a video signal into encoding-resultant video datadivided in units; adding an audio time stamp for audio-video synchronousreproduction to every unit of the encoding-resultant audio data togenerate time-stamp-added audio data; adding a video time stamp foraudio-video synchronous reproduction to every unit of theencoding-resultant video data to generate time-stamp-added video data;multiplexing the time-stamp-added audio data and the time-stamp-addedvideo data into main data; to first after-recording-purposeencoding-resultant data for at least one of (1) the encoding-resultantaudio data and (2) the encoding-resultant video data which form the maindata, adding a time stamp for reproduction synchronous with a portion ofthe main data and identification information for identifying the firstafter-recording-purpose encoding-resultant data to convert the firstafter-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and recording the maindata, the second after-recording-purpose encoding-resultant data, andplay list information on a recording medium, the play list informationproviding link between the main data and the secondafter-recording-purpose encoding-resultant data.
 3. A method oftransmission, comprising the steps of: compressively encoding an audiosignal into encoding-resultant audio data divided in units;compressively encoding a video signal into encoding-resultant video datadivided in units; adding an audio time stamp for audio-video synchronousreproduction to every unit of the encoding-resultant audio data togenerate time-stamp-added audio data; adding a video time stamp foraudio-video synchronous reproduction to every unit of theencoding-resultant video data to generate time-stamp-added video data;multiplexing the time-stamp-added audio data and the time-stamp-addedvideo data into main data; to first after-recording-purposeencoding-resultant data for at least one of (1) the encoding-resultantaudio data and (2) the encoding-resultant video data which form the maindata, adding a time stamp for reproduction synchronous with a portion ofthe main data and identification information for identifying the firstafter-recording-purpose encoding-resultant data to convert the firstafter-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and transmitting themain data, the second after-recording-purpose encoding-resultant data,and play list information via a transmission line, the play listinformation providing link between the main data and the secondafter-recording-purpose encoding-resultant data.
 4. An after-recordingapparatus comprising: first means for encoding a first audio signal intofirst encoding-resultant audio data; second means for encoding a videosignal into encoding-resultant video data; third means for adding afirst audio time stamp for audio-video synchronous playback to the firstencoding-resultant audio data generated by the first means; fourth meansfor adding a video time stamp for audio-video synchronous playback tothe encoding-resultant video data generated by the second means; fifthmeans for multiplexing the time-stamp-added audio data generated by thethird means and the time-stamp-added video data generated by the fourthmeans into main data; sixth means for recording the main data generatedby the fifth means on a recording medium; seventh means for encoding asecond audio signal into second encoding-resultant audio data; eighthmeans for encoding a third audio signal into third encoding-resultantaudio data; ninth means for adding a second audio time stamp foraudio-video synchronous playback to the second encoding-resultant audiodata generated by the seventh means, the second audio time stamp beingequivalent to the first audio time stamp regarding audio-videosynchronous playback; tenth means for adding a third audio time stampfor audio-video synchronous playback to the third encoding-resultantaudio data generated by the eighth means, the third audio time stampbeing equivalent to the first audio time stamp regarding audio-videosynchronous playback; eleventh means for recording the time-stamp-addedaudio data generated by the ninth means and the time-stamp-added audiodata generated by the tenth means on the recording medium; twelfth meansfor generating first play list information providing link between themain data and the time-stamp-added audio data generated by the ninthmeans; thirteenth means for generating second play list informationproviding link between the main data and the time-stamp-added audio datagenerated by the tenth means; and fourteenth means for recording thefirst play list information generated by the twelfth means and thesecond play list information generated by the thirteenth means on therecording medium.
 5. A reproducing apparatus for use with a recordingmedium storing main data, a plurality of after-recording-purpose audiodata, and a plurality of play list information, the main data includingmain video data and main audio data, the main video data containing avideo time stamp, the main audio data containing a first audio timestamp, the video time stamp and the first audio time stamp being forsynchronous playback of video and audio, the plurality ofafter-recording-purpose audio data containing second audio time stampsrespectively, the second audio time stamps being equivalent to the firstaudio time stamp regarding synchronous playback of video and audio, theplurality of play list information providing link between the main videodata and the plurality of after-recording-purpose audio datarespectively, the reproducing apparatus comprising: first means fordesignating desired one among the plurality of after-recording-purposeaudio data in response to user's request; second means for reading outthe plurality of play list information from the recording medium; thirdmeans for identifying the desired after-recording-purpose audio datadesignated by the first means in response to the plurality of play listinformation read out by the second means; fourth means for reading outthe after-recording-purpose audio data identified by the third meansfrom the recording medium; fifth means for reading out the main datafrom the recording medium; sixth means for separating the main data readout by the fifth means into the main video data and the main audio data;seventh means for detecting the video time stamp in the main video datagenerated by the sixth means; eighth means for detecting a second audiotime stamp in the after-recording-purpose audio data read out by thefourth means; and ninth means for synchronously playing back theafter-recording-purpose audio data read out by the fourth means and themain video data generated by the sixth means in response to the videotime stamp detected by the seventh means and the second audio time stampdetected by the eighth means.
 6. A reproducing apparatus for reproducingafter-recording-purpose data and a portion of main data from a recordingmedium, the recording medium storing the main data, theafter-recording-purpose data, and play list information which arerecorded thereon in a procedure comprising the steps of compressivelyencoding an audio signal into encoding-resultant audio data divided inunits; compressively encoding a video signal into encoding-resultantvideo data divided in units; adding an audio time stamp for audio-videosynchronous reproduction to every unit of the encoding-resultant audiodata to generate time-stamp-added audio data; adding a video time stampfor audio-video synchronous reproduction to every unit of theencoding-resultant video data to generate time-stamp-added video data;multiplexing the time-stamp-added audio data and the time-stamp-addedvideo data into main data; recording the main data on the recordingmedium; adding, to first after-recording-purpose encoding-resultantdata, (1) a time stamp for reproduction synchronous with a portion ofthe main data and (2) identification information for identifying thefirst after-recording-purpose encoding-resultant data to convert thefirst after-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and recording thesecond after-recording-purpose encoding-resultant data and play listinformation on the recording medium, the play list information providinglink between the main data and the second after-recording-purposeencoding-resultant data, the reproducing apparatus comprising: firstmeans for reading out the play list information from the recordingmedium; second means for reading out desired main data and desiredsecond after-recording-purpose encoding-resultant data from therecording medium in response to user's request by referring to theread-out play list information; third means for separating the read-outdesired main data into desired main video data and desired main audiodata; and fourth means for synchronously reproducing contents of thedesired main video data and contents of the desired secondafter-recording-purpose encoding-resultant data in response to timestamps therein.
 7. A computer-readable medium storing a computer programfor reproducing after-recording-purpose data and a portion of main datafrom a recording medium, the recording medium storing the main data, theafter-recording-purpose data, and play list information which arerecorded thereon in a procedure comprising the steps of compressivelyencoding an audio signal into encoding-resultant audio data divided inunits; compressively encoding a video signal into encoding-resultantvideo data divided in units; adding an audio time stamp for audio-videosynchronous reproduction to every unit of the encoding-resultant audiodata to generate time-stamp-added audio data; adding a video time stampfor audio-video synchronous reproduction to every unit of theencoding-resultant video data to generate time-stamp-added video data;multiplexing the time-stamp-added audio data and the time-stamp-addedvideo data into main data; recording the main data on the recordingmedium; adding, to first after-recording-purpose encoding-resultantdata, (1) a time stamp for reproduction synchronous with a portion ofthe main data and (2) identification information for identifying thefirst after-recording-purpose encoding-resultant data to convert thefirst after-recording-purpose encoding-resultant data into secondafter-recording-purpose encoding-resultant data; and recording thesecond after-recording-purpose encoding-resultant data and play listinformation on the recording medium, the play list information providinglink between the main data and the second after-recording-purposeencoding-resultant data, the computer program comprising the steps of:reading out the play list information from the recording medium; readingout desired main data and desired second after-recording-purposeencoding-resultant data from the recording medium in response to user'srequest by referring to the read-out play list information; separatingthe read-out desired main data into desired main video data and desiredmain audio data; and synchronously reproducing contents of the desiredmain video data and contents of the desired secondafter-recording-purpose encoding-resultant data in response to timestamps therein.